FORM FOR RECORDING ASSESSMENT INSTRUMENT PLAN 



Abbreviated , 
Objective 


Learning Behavior 
to Be Assessed 


Assessment Instroments 


1 


• 




2 






3 






4 


■ 


0 

4 

'■ 


3 







7. DEVELOP DATA COLLECTION PROCEDURES 



Definition: DATA COLLECTION-The act of obtaining data from or aboot program 


)articipantsb} means of a specified 


procedure or set of procedures.. . 





> 

Assessment Instrument 


■ ' EXAMPLE 
Data Collection Procedures 


Date 


Vil Achieveme'.t Test 


1 , Training staff and faculty in test administration 
scoring procedures 


Sept, 25 




2, Select random saiiiple of students to be assessed 


Sept, 10 




3, 'Administer the test according to the standardized 
instructions in the test manual 


Oct, 5-16 




4, Monitor testing and assemble completed 
instruments 


Oct. 5-16 



WHAT ARE YOU EXPECTED TO DEVELOP FOR THIS SECTION?^ > 
■» A list of data collection procedures for each as.sessment instrument 



i5 

100 « 
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' * . INSTRUCTIONS 

1 . Refer lo the previous section on Identifying Assessment Instruments, page 35, List the instruments you have selected 
on the form on page 42 under the appropriate category (i.e., achievement tests, observation instniments. question-., 
tiaircs, checklists/logs), 

Check to\sce if there is a test administration manual for each instrument, 

' ' ' . , , 

If there is a'niiinual available for any of the instruments, determine whether standard instructions are available for 

adii]jfii.stering the instruments and scoring the results. Check the appropriate columns for each instrument. 

2. If there are standard instructions for administering and scbring^^ assessment instruments you will be using, label 
each lesi administration manual with the number of the program objective to^be assessed (i.e.' [irogram objective I). 
Have the manuals available for future reference, ^ 

-J. 

3. Determine when the assessment instmments will be used to collect data and who will be responsible for collecting the 
data (who will be administering the instniments). Record, this information on the form on page 43, ^ 

After completing the form, proceed to the next section on page 45, 

4. !f there is no manual of instructions for administering a selected or developed instrument, or if theiiwtructions are 
incomplete, you will have to develop a set of instructions. Use the guide below to assist you in developing a set of 
procedures, ■ , 

• A precise set of directions for administering the instalment 
I A precise set of directions for scoring the instrument 

• Specification of the population for which the instrument is intended 

• Specification of time required to administer the instrument 

• Specification of the conditions required for the administration 'of the instrument such as room environment, 
groupim:, etc. 

After you have drafted a set of data collection procedures, have someone in your program review them to see if the 
procedures are clear and concise. Make revisions if required. 

Recora data collection procedures, dates, and personnel on page 43, 
After completing the form, proceed to the next section on page 45, 



WHAT YOU ACCOMPLISHED IF YOU COMPLttED^THIS SECTION 

Determined if stai. -rd instiuctions are available for selected assessment instruments. 
Developed a set of instructions if they were not already available 
Recorded data ci^llection procedures, dates, and personnel 



FORM FOR LISTING ASSESSMENT INSTRUMENTS 

•'I 

List (he assessmeni initrunients you selected to evaluate year "programs and place a check in.the appropriate colamn. 



1 

Assessment Instruiiiciits 


Standard Instructions 
Are Available for; 


Administration 


Scoring 


Achievement Tests-Norm-Re ereiiced 

L 

I ' ' 

I. 

) 

4. . ' . 


t 

•1 • 


r 


1. ■ ■ , 




4 

\ 

S 


V/L'r>tl V UUUU 1 11 HI UlU^ 111.1 

1, . 

1 ' ■ 

if 

^ 

4. ; / / 


• 


/ 


Qiicslionnaire 

1, 
v 

3, ' '■ ■ ' * • . 

4, : "■ ' ' . 






Checklists/Loils 

1. • ■ ; ^ 
3, , 

V. ' 




1 

1 



FORM FOR RECORDING DATA COLLECTION PLAN 



Assessment Instrument 
and Objective 



Data Collection Procedures 



Dates 
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8. SELECT DATA ANALYSIS TECHNIQUES 



Definition: DATA ANALYSIS TECHNIQUE-A proa 
dispby, 


;dure applied to a set of data for purp 


oses of summa 


rization or 





EXAMPLES 

"BASIC DESCRIPTIVE STATISTICS 

■."'.'.,,1 

• Arithmctic'\iiean~coiiipiitini; the average score 



• Median-computing the midpoint in a range of scores 

• Standard deviations-coiiiputinsi the variability within a group of scores 
MORE COMPLETE INFERENTIAL STATISTICS 

• t-test-coniparison between two means to tell whether they differ significantly 

• Analysis of variance-comparison of several arithmetic means to determine whether the means differ significantly 

WHAT ARE YOU EXPECTED TO DEVELOP FOR THIS SECTION? 
I A recorded data analysis technique for each program objective 



45. 



INSTRUCTIONS 

1 . Reread evaluation section of your program docoment (if yoo have one), Determine whether data analysis techniques 
have been- specified. 

2. If data analysis teii:l;iniques have been specified, determine whether they are appropriate for each program objective 
and accompanying evaluation design you selected on page 27 in this workbook. Refer to pages 48 and 49 in this 
workbook to determine the appropriateness of your techniques, 

0 

3. If the specified data analysis techniques are appropriate for each program objective, turn to page 47 in this workbook 
and record them, 

4. If data analysis techniques are not specified or are inadequate, you will have to select appropriate ones. Refer to the 
chart on pages 48 and 49 in this workbook for information that will help you in making an appropriate selection. 

5. Record the data analysis techniques you have selected on the form on page 47, 

6. Proceed to the next section of page 51, 

WHAT YOU ACCOMPLISHED IF YOU COMPLETED THIS SECTION 
• Identified whether diila iiniilysis tcchni^L|Lies had been specified 

I If they had not been specified, you were able.io select appropriale analysis techniques based on the program objectives 
and evaluation design. 
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FORM FOR RECORDING DATA ANALYSIS PLAN 



List all the data analysis techniques selected to measure each program objective. There may be more than one technique for 
each objective if you are using more than one assessment instrument, 



Abbreviated 
Objective ' 


Assessment 
Instrument 


Selected Data Analysis Technique 


1 






2 




1 1. 


3 


- 


' 0 ^ 

•' * 

A 


4 




t ■ ; . 

\ . ^ ■ , 
— p^Jp ■ i.ii ■ ^- m — ' — ■■ ■■■ I" ■■■■■ 


* 
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ADDITIONAL INFORMATION 



RECOMMENDED DATA ANALYSIS TECHNIQUES 
FOR SELEQED EVALUATION DESIGNS AND OBJECTIVES 



Evaluation Dcsicns 



Example of Objectives 
Specifying an Evaluation Design 



Recommended Data 
Analysis Techniques 



Evaluation Designs Using the Special 
Program (Experimental) Group as a 
Standard 

] . One Group Pretest-Posttcst 



Evaluation Designs Using a Crilerion for 
\[ Standard 

i. Individual Student Criterion 
. Mastery Design 



Remedial reading students with 80 per- 
cent or better attendance will show a 
mean gain of 4 months in reading com- 
prehension for every 4 months of instruc- 
tion. Gain will be measured by the 
CTBS, Level 3 reading comprehension 
section. 

The median percentile rank for those 
students at Y School:cnrolled in the K, 
1st, 2nd, 3rd gradc[>;feiiding programs 
will be 5 points higher for each grade 
on a May 1 975 posttest than on the Oc- 
tober 1974 pretest for the same stu- 
dents, j . 



Every student will correctly respond to 
75 percent of the items on the Reading 
Mastery Test given at the completion of 
Reading Unit A. 



Derive the arithmetic mean— a descrip- 
tive statistic (see statistics book or a con- 
sultant for the computational formula) 



Derive the median— a descriptive statis- ■ 
tic (see statistics book or a consultant for 
the computational formula) 



Calculate a percentage— a descriptive 
statistic (.Tee stati.stics book or. a consul- 
tant for the computational formula) 
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Evaluation Designs 



3, Group Criterion Mastery Design 



Evaluation Designs Using Comparison 
Groups as a Slandard 

4, Pre(es(-Post(est Design with 
Comparison Group (random as- 
signment of students to both groups) 



i Posltest Only Design with Com- 
parison Group (random assign- 
ment of students to both groups) 



6. Prelesl-Posttest Design willi 
Nonequivaicnt Comparison 
Group (students have not been ran- 
domly assigned to groups) 



■ Example of Objectives 
Specifying an Evaluation Design 



Prior to February 1975, 80 percent or 
more' of the full-time trainees enrolled in 
Graphic Arts in September 1974 will 
complete 12 or more of the instructional 
components, 



Upon completion of the second year of 
the experimental math program, students 
participating in the program will show a' 
statistically significant gain (.05) level 
from pretest to postlest on the Iowa 
Mathematics Series over those students 
assigned to the regular math program, 

Students completing 3 years of the 
Follow-Through Program will score sig- 
nificantly higher (.05) level on 'the Il- 
linois Test of Psycholinguistic Abilities 
than students who received the regular 
school program for 3 years,' 

After 8 months of instruction, lOth grade 
students enrolled in the Work Experi- 
ence Program will show a statistically 
significant gain (,05) level on the Florida 
Social Studies Inventory as compared to 
the scores made by the control group. 



Recommended Data 
Analysis Techniques 



Calculate a percentage-a descriptive 
statistic (see statistics book or a consul- 
tant for the computational formula) 



Derive the "t" test on mean posttesi 
scorc'S-an inferential statistic (,see 
statijiitics book or a consultant for the 
conptational formula) ' ; 



Derive the "I" lest- on mean postlest 
score.s-an inferential statistic (see 
statistics book or a consultant for the 
computational formula) 



Derive the "t" test on change scores- 
'an inferential statistic (see statistics book 
or a consultant for (he computational 
formula) 



■its 
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9. PLAN f pR THE PROCESSING OF DATA 



Definition: PROCESSING OF DATA-The ^ysleipalic ireaJmeni of collecled data according to specified procedures 



EXAMPLE 

XYZ Achiev.'nieni Test • Scoring on the primary form to be done by hand 

I Grade equivalencies and percentiles for the primary form to be computed by hand 

* 

I Scoring on elementary form io be done by computer center 

• Grade equivalencies, ;;.id percentiles for the elementary form to be done by computer 

I All statistics to be run on the computer 



WHAT ARE YOU EXPECTED TO DEVELOP FOR THIS SECTION? 
, • A recorded plan for Ihc processing of data 



0 



51 



INSTRUCTIONS 

I. Check k evaluation section of your program document (if you tiave one) to determine whether k plan has been 
developed which specifies; (I) liow the data from the assessment tools will be processed (including scoring, tabula- 
tion, conversion of raw numbers), (2) how the' data analysis techniques will be applied, and (3) who will be 
fcspnnsible for processing the data, 

I. If a plan has bc.Mi developed, record it on the fori! on page 53, . , 

3. If there is ho pLiii for liic pr kissing of the data to be gathered by the 'assessment instruments, you will have to develop 
one. 

, Turn to the recording from on page 53, List your assessment instruments (use abbreviations where necessary), 

Determine what steps will have to be performed in order to process the data your instruments have provided (see the 
example on page 51), This will include the scoring and tabulation of information responses and how computa'.ions are^ 
to be performed. Record this information on the form.(page 53), 

Now determine who will perform the ,'ibovc steps. Will itk classroom teachers? The evaluator? Clerical staff.' A 
test publisher scoring service? Record the information on the recording form (page 53). 

4. Proceed to^hc nc\l section on page 53. 

WHAT YOU ACCOMPLISHED IF YOU COMPLETED THIS SECTION ' 

■ . . ■ \ 

I Identified whether a plan for processing data had been developed 

• If no plan had been developed, you were able to develop a plan for processing data gathered from assessment 
instruments and for computing using specified data analysis techniques. 



FORM FOR RECORDING DATA PROCESSING PLAN 



Abbreviated 
Objeeiive 


Assessment Inslrunienl 
(Abbreviate) 


Steps Required to Process Data • 
(see example on page 51) 


Persons Responsible 
for Processing Data 


Objeeiive 1 


































Objeeiive 2 












! 




















Objecliw } 

1 
1 


















■ 














Objective \ 




..- 




























Objeeiive 5 
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10. DEVELOP PROGRAM-MONITORING PROCEDURES 




In the previous seclions, you have been focusing upon the program objectives and on determining appropriate design and 
procedures. Now you will focus upon the program activities which have been -designed to accomplish the program 
objectives and determine how you will plan to monitor those activities, / 





EXAMPLE 




Activity 


Monitoring Procedures 


' Dates 


Implementation of the Wilcox 
Reading Program 


Monthly classroom visitation 
using Monitoring Form 2 
as a recording device 


Monthly 
Oct-May 



. WHAT ARE YOU EXPEQED TO DEVELOP FOR THIS SECTION? 

• A recorded plan for the monitoring of your program activities 

• Formstouse when monitoring those activities 



/ 



INSTRUCTIONS 

1 . Reread the section of your program document which specifies what activities will be conducted for each program 
objective and determine whether a plan has been developed fo'r rnonitoring Mt activities during the year, 

e 

2. If a monitoring plan has been developed, record it on the form on page 57. If there are many columns left blank, 
iietermine the information reqiiired and fill in the missing data. 

3. If a monitoring plan has not been developed, determine what activities you will want to.monitor during the operation 
of the program and list them on the recording form on page 57 in this workbook.. 

4. Determine what types of monitoring forms you will be using to collect information. There are samples of monitoring 
forms starting on page 59 in this workbook. You may want to adopt or adapt some of them or develop your own forms. 

Indicate the forms you will be using by filling out the second column on the recording form on page 57. 

5. Determine whqn monitoring will take place and record the information on page 57. ' 

6. Finally, you will want to determine who will be doing the monitoring. Indicate those persons in the last column on the 
recording form on page 57. , 

7. After completing this section proceed to the next section on page 65. ' ; .. 

WHAT YOU ACCOMPLISHPD IF YOU COMPLETED THIS SECTION 

• Determined whether a monitoring plan enlisted for your program .) 
« If it did not exist, you were able to develop a plan 

• Selected monitoring forins to use in implementing the monitoring plan 
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FORM FOR RECORDING MONITORING PLAN 



Activities 
to be Monitored 



Observation Instrument 
to be Used 



Monitoriikg Dates 



for Monitorin" 



0 



5] 



'139 



SAMPLE MONITORING 
FORMS 



V 



ERIC 



If V 4f 



59 



Program Activiiies 



SAMPLE MONITORING MM 1 

Is Aclivky Taking Evidence Available . 

Place aiS Planned? . • » 

Nol ' A ■ 

Yes' No Known ' Observation Records Conference Olhci 



None' 





D 


D 


0 




D 






^ D 


> (LislAcliviiies to Be Monitored) 




















• D 


D 




D 


• D. 


. D 


,D 


D 


1 


D 


D 


Q 


/ □■■ 


D 


D 


n 


■ D 






0 














4 






D 




D 


D 


D 


■ D 






D 


D 


'. D • 




■ D 


D 


D 


fi' ' 


C' 




D 


D • 


0 




"D 


P 


v. 


•d: 


D 


D 


D. 


D * 




D 


,D 


r' ■ 


D 




„ D ■ 


D • 


□ 






■D 


V 


D 


D 


D 




D 






D 


in 


D 






' D 


€ 


D 


■ □ ■ 


D 




□ 


D 


a ■ 


< ■"■D 


U 




u 


D 



'>2 



f, ' 
i 



1" 



00 



EXAMPLE OF HOW TO USE FORM ! 



Program Activities 



Is Activity Taking 
Place as Planned? 



Evidence Available 



Reading instruction in Spanish 
will be given to each child 
each day for 20 minutes, 

2. 'Each classroom will be respon- 
sible for developing 5 language 
■ centers.^ • ■ ' 

3: Parents will be encouraged to 
nijike one classroom visit.per 
month to observe their child's 



Not , ' • 

•Yes No Known Observation Records ^ Conference Other None 



D D D 



Note: The above activities are not all instructional, Activities 2 and 3 are program-support activitii's, 
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Instniclional Activities 



/ ■ 

SAMPLE MONITORING FORM 2 



Time Allotted 
per Activity 



2. 

3. 

4. 

•5. 

6. 

7. 1 

8 

9. 

m ' ^ 

II 

12.: 



T7- 



Aclivity Conducted 
according to Pro- 
gram Specifications 



Yes 


No 


□ 


□ 


□ 


□ 


□ 








□ 




□ 


□ 


□ 




□ 


□ 


□ 


□ , 


□ 


□ 




□ 


□ 


□ 


□ 


□ 



63 



EXAMPLE OF HOW TO USE FORM 2 



Instructional Activities 



Time Allotted 
per Activity 



Activity Conducted 
according to Pro-' 
gram Specifications 



Yes • 



No 



1, Reading instruction in Spanish will be given 20'Min/Day 
to each child, each day for a period of 20 inin. , 



2. Teachers will instruct their class in the 
folklore of Mexico. 

3. ' instruction in math will be given to each 

child in his/her dominant language. 



45 Min/Week 



15 Min/Day 



D 



D 



□ 



□ 



D 



Note: All of the abovfc activities are inslruclional. It would not be appropriate to include program-support activities in this element. 
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Hi DEVELOP REPORTING PLAN 

f 



Definition: REPORTING-The supplying of evaluation informationjo decision makers and community members 



/ 



. TYPES OF REPORjriNG^ 
' ' ' ■[ t . ' . ■ « 

Ijitcrim Reportlng-Reporting^hat takes place during the program, Student progress and/or status of the implementation 
of program activities is reported. 

End-of-the-Year Reporting-Reporting that takes place at the end of the program. Results of the program are compared 
lo program objectives. ' " ' 



I 



EXPECTED PRODUCT 



• A recorded plan for reporting evaluation information 



ERIC 
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\^ INSTRUCTIONS 

■ ' • *\ 

1 . In this workbook, on pages 23 and 24, you' have rei-q^rded a list of data requirements to be used in the decision-making 
process. Determine wliicli requirements will necessitate a report, either oral or written. 

List the data to be reported on the I^orm for Recording Your Reporting Plan od page 67. Remember to look for 
duplications in the data required, You will list the requirement only oncx on the Reporting Plan form, 

2. Add the dates to the Reporting Plan form as to when the data is to be required, 

3. List oil the P.eporting Pliiii form all those persons requiring the information, 

• Board members 

• Superintendent 

• Administrators . ' " • ' , 

• Instructional staff 

• Parent/community advisory committee members : ' 

• Students ; 

• Community agents, newspaper editors, etc, 

4. Determine how you will be reporting the evaluation information,, Check one or both of the columns under "Mode of 
Reporting," . . 

5. After you have coilipleted the Reporting Plan form verify the data required and ibe daies with each of the persons 
designated to receive the information. . 

6. Remember to obtain district clearance on all reports before they are disseminated. . ■ 

7. Turn to ihe ne,\t .section on page 69, 

' WHAT YOU ACCOMPLISHED IF YOU COMPLETED THlS SECTION 
« Developed a plan for reporting evaluation information. 
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FORM FOR RECORDING YOUR REPORTING PLAN 

• ^ \ 



•> 

Evaluation Data to Be Reported 


Who Is 
Responsible 
' for Report? . 


Mode of 
Reporting. 


. Report 
Format 
Developed? 


When 

IS 

Report 

. Due? 


Whn Will Rpcpivp 

tE^Report? 

■ \^ 


What Will the 
Report Data Col. 
Be Used for? 


Written 


Oral 


Yes 


No 


I' 


\. 

1 


1 

} 


> 








\ 

\ 

\ 

■ • ' \ 
$ 


/ 

\ 

f 



67 



12. DEVELOP TIMELINE TO MANAGE 
THE EVALUATION ACTIVITIES 



.Deflnition: TIMELINE-A management tool which graphically shows the functions/tasks to be performed, their time 
sequence, and the amoun! of time estimated for completion of each functiian or task 



EXAMPLE 



• f 

lilonths 

. .Management Functions 1 2 ' 3 4 5 


1 

0 

73 
3 

> 

E 

u 

% 

0 

' u 


Program Director 


Instructional 
Staff 


-Others " 


Select evaluation staff 


i- 










X 

L 


X 






Order materials 


k 


-k 








X 


^x 







WHAT ARE YOU EXPECTED 10 DEVELOP FOR THIS'SECTION? 
^ A maiiagemeiif tihieline 



INSTRUCTIONS 

1. N Now that you have almost Onished with planning for the evaluation of your program, you will want to develop a 

timeline for managing the major evaluation activities. Check ycir program document (if you have one) to see whether 
a management timeline for conducting the evaluation has already been developed. 

2. !f a timeline has been developed, review ifto determine whether it includes all of the activities that will be important to 
specify for i' ^ling the evaluation. .Mso, check to .see whether the dales and personnel are correct according to yoor 
evaluation . , ' 

If the tiiiidiii, sains adequate afte^ this review, pull it out of the program uocument and put it up on a wall where you 
laii ret'.! hi II \ci\\k\\s. 

3. If Ml iiiagenient timeline available, you will want to develop one. Since each program evaluation will be 
somculiii HK|iic, you will have to construct your own timeline. On page 72 is a sample of a timeline and how to fill it 
out. Use this as a guide for developing your own. On page 73 is a timeline form which you cm use to record your 
manasemenl timeline, 

Oil page 74 of this workbook, you will find a list of common evaluation-management functions. Select thusr functions 
which apply to your evali.i;i!ion and list them on the timeline form on page 73. You will want to add some of your own 
functions which are noi li.stcd here. ^ .■ ' 

After you havetsied the functions in sequential order, indicate the span' of lime during which each activity will take 
place and note the completion date. For the former, use the following code; 

A Function to occur on that date _ , 

■ Itk Function wiH take place over a period of time (e.g., October 9 - October 15) 

Lkvt all the personnel involved in the evaluation activities in the spaces provided for personnel at the lop of the 
timeline. For example: . , , ■ — . '» 



Check the iippropriatc boxes after each management function which indicate the persons invqlved'in iha' function; 





















c 

c 










u n 


■ M 

■ u 




U 




u 3 

2 > 


0 u.' 


a 
\m 

to 




W 


Miinagement Functions • 


D. a] 


^ -r 


C 




0 


Prepare interim reports 


X 








Obtain staff services 


X 


. X 







4. Rj\ cw the limelinc with the appropriate staff members to make sure you have irieludd ail of the major ev;iiuat!on 
(iiii'iioiis. Revise thj timeline if changes are required. 

If the tinjeline seems adequate after this review, pull it uni of the program document and put it up on a wall where you 
can refer to it regularly,, 

5. Proceed to the next section of this handbook on page 75. 

WHAT YOU ACCOMPLISHED IF YOU COMPLETED THIS SECTION 

• Determined whether an evaluation timeline existed for your program 

• If there was m) evaluation timeline, yoii developed one. 
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EVALUATION TIMELINE 



' ■ Time When the Activity . 

WillOccilr 

/ 1 . ■ 

Months 

(July to July) : 
tion ■ ^ , 

Management Funclions ' Date 1 2 3 4 5 6" 7 ■ 8 9 lO' II 12 


u 
0 

!1 
3 

CI 
> 

w 

c 
c 

u 

bO 
0 
u 

a, 


j Program Director | 


Urn 

C 

M 


w 

a 

a 
c 
t 

H 


4; 
•0 

< 
X 

« 

h, 


2. 
w' 

13 
(J 

'C 
u 

U 


UDlilin CVdiUaUUIl SIull/MlVllCs 


9/6 






A 
m 




















X 


X 










Order commercial assessment tools 


Q/K 




A 

A 






















X 












Adapt nonstandardized assesr.ient tools 


O/i/ 


A- 


A 

-A 




















/ 


X 












Reproduce adapted tools 


7/1 




i 


i 




















■ 










X 


Develop data collection'procediires 


Wo 




A 


m 




















X 












Train data collectors 


y/iu 




















/ 






X 












Select student assessmen! sample 


9/22 






A 

A 




















X 












Complete data collection schedule 


9/28. . 






▲ 




















X 


Y 
A 


X 


X 


X 




Administer pretest tools 


10/14 








i 
















) 


X 




X 


■ 


X 




Elc. ; 
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FORM FOR RECORDING EVALUATION TIMELINE 



Time When the Activity 
Will Occur 

Months 

* , Comple- 
tion ! 

Management Functions Date 1 2 3 4 5 6 7 8 9 10 11 12 


Program Evaluator 


Program Director. 


Instructional Staff 


Principals 


Teacher Aides j 


Clerical Staff 






\ 






































































































\ — 


























































































































































































































































































































































































1 


1- 
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ADDITIONAL INFORMATION 



s6me evaluation-management functions 



Esubliiih program go[ils 

Establish progr:;nu)bjectivc,s 

Determine needs , ■■ 

Obtain staff services m 

Order commereiaLassessmcnt instruments 

Adapt iionstandardized assessment instruments 

Oevdiip data collection procedures for local instriinients 

Select student assessment samples ' 

Cons'iicl inservice trainiiig 

Q-mpiete data coilect'on schedule 

Repri'uce adapted ins'rjmentj; 



Train data collectors ^ \ ■ 
Administer pretest insrumenis 
Score/tabulate data ' 
Analyze data , 
Monitor program activities 
Prepare interim reports 
Submit interim reports , , ■ 
Administer posttcst instruments , • 
Develop evaluation plan for next year 
Prepare summative report' 
Submit summative report 



Kl.EP A RECORD HERE OF ADDITIONAL MANAC '^iNT FUNCTIONS 
\' aDE PART OF YOUR PROGRAM EVALUATION 'lASK . ^ ' 



( 



13. IDENTIFY RESGIRCES REQUIRED FOR THE EVALUATION 




TV-RHS OF^RESOURCES 

• Personnel 

• . .Materials and Supplies 

• ■ Equipment 

• Services 
« Space ■ 

• Time 

• , Money 



WHAT ARE YOU EXPECTED TO DEVELOP FOR THIS SECTION? 



• "A li.,. of resources needed to conduct evaluation plans that are within your budgetary allotments 



\ ■ 
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INSTRUCTIONS 

I; Review your evaluation plan as developed in this' workbook. List the evaluation resources and dollars required to 
implement your evaluation plan. Use the form on page 77 to record this information. ■ • 

2. After listing required resources and dollars, add the figures in tlie columns and enter totals in the spaces marked "Total 
■ $ Required," ^ 

3. Refer lo your evaluation budget (if you have one) and locate the amounts budgeted for each of the resource categories 
(personnel, maierials, equipment, services), 

Insert the dollar figures you have available in the spaces marked "Total $ Available," 

, - I ' 

4. Now add the "Total $ Required" in the four columns, Write that figure in the space marked "Grand Total Required." 
■ , Required," ^ 

Add the "Total S Available" in the four columns. Write that figure in the space marked "Grand Total Available." 

1 ' ' 

Subtrai'! those grand total figures. Record the difference in the space marked "$ Diffecence," 

If yi)ii arc iihin your allotted budget, congratulations! You will not have to make any changes'. Your final planning step 
will be 'to obtain approvals on your total plan. 

If you are over your allotted-budget amount, you will have to resolve the difference,, One way tb resolve the budget 
difference is to determine whether it is possible to get more money for your evaluation, If you do get^ntore money and it is 
enough to cover the total cost of your required resources, you may now proceed to page 79, 

If you'cannot get any more money for your evaluation, you must now begin to modify your evaluation plans. 

Consult with your program staff and other appropriate personnel to help you inake revisions and budgetary cuts. 

WHAT YOU ACCOMPLISHED IF YOU COMPLETED THIS SECTION 
• Analyzed and listed the resources required to conduct an evaluation . 



FORM FOR RECORDING RESOURCE PLANNING , 



Evaluation 
Personnel 


$ 


Materials 


$ 


Equipment 


$ 


Services 


'$ 


^ 
















Total $ Required 




lotal $ Reijuired 




Total $ Required 




Total $ Required 




Total S Available 




Total $ Available 




Total $ Available 




Total $ Available 





Grand Total Required. 
Grand Total Available, 
$ Difference 



14. SUMMARY AND WALL CHART 



You have now developed an evaluation plan, if you completed this workbook you were helped to: 



\ 
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Define the purpose of the ^valuation 

Review program needs assessment information 
^ Review program goals and objectives 

Identify data requirements 

Determine evaluation design 

Identify assessment instruments 
' ' Develop data collection procedures 

Select data analysis techniques 

Plan for the processing of data 

Develop program monitoring procedures 

Develop a reporting plan 

Develop a timeline 

Identify required resources 



The wtlkhart'accoinpanying this workbook will serv^ as a basic management tool during the implementation of the 
"evaluation plan, The next pages provide instructions in how to fill it out. 



Final note; 

If you have done a thorough job of completing each section of this yiorkbook, it, too, will prove to be a valuable 
management tool. Frequent reference to the plans you recorded in the early stages, as those plans materialize, will 
help keep the program evaluation on track. In a very real sense, the Workbook on Program Evaluation, once 
completed, is the program evaluator's tool for monitoring his or her own performance as a program evaluator. It 
will prove valuable also as you start afresh to pursue the next cycle to enhance the quality of education through 
continuing program improvement. 
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INSTRUCTIONS FOR FILLING OUT THE WALL CHART 

You'have already completed planning and recording the evaluation for your program. Key informafion which will assist 
you in implementing your plan is now to be transferred from this workbook io your wall chart, 

♦ 

The wall chart contains 1 1 columns. The information to be v, ritten in those columns can be found in this workbook on the 
pages indicated below: 



I Statements of Need 

I Program Objectives 

I Evaluation Design , 

I Assessment .Tools 

I Assessment Dates & Personnel ... , 

I Data Analysis Techniques '. . 

• Program Activities to be Monitored 
I , Monitoring Dates & Personnel' ... . 

I Key Reporting Dates : 

I Persons/Agency to Receive Reports 

• Use to be Made of the Data/Report 



...Pages 12, 13 
Pages 18, 19,20 
..Page 27 
..Page 38 
. . Page 43 



..Page 57 
..Page 57 
. . Page 67 
..Page 67 
, . Page 67 



Page 81 provides you with a sample wall chart with partial entries. Use it as a guide for making and 
filling out a wall chart of your own. 
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SAMPLE OF A PARTIALLY COMPLETED WALL CHART 



STATEMENTS 
OF.NEED 



Need: 

To \mm reading 
opraprehcnsion skills 
of 3rd &4ih grade 
students so ihat they 
will be at grade level 
by the end of the 4tb 
instruclional year. 



PROGRAM 
OBJECTIVES 



75%orihe3idgr&de 
students pmidpating 
in the ABC Reading 
Program will score at 
gr}de level or above 
ontlicaeS Read- 
ing Comprehension 
sublet given after 8 
months of 
specialized instniC' 
(ion _ 



I(}0<lofthe4th 
grade students partiC' 
ipating in tli>: ABC 
Rciding program 
for at least 2 years 
will hcore at grade 
level ontheCTBS 
Reading Comprchen' 
sion subtest. 



EVALUA. 

TION 

DESIGN 



Group 
Criterion 
Mastery 
Design 



Group 
Criterion 
Mastery 
Design 



ASSESSMENT 
INSTRUMENT 



Comprehensive Test 
of Basic Skills 

Teacher-made in* 
terim criterion- 
referenced test 



Comprehensive Test 
of Bisic Skills 

Tcachei'inade in- 
terim criterion- 
referenced test 



ASSESSMENT 
DATES AND 
PERSONNEL 



May 

Evaluator & 
Aiisislant 

September 
January 
Classroom teachers 



September 

May 
January 



DATA 

ANALYSIS 

TECHNIQUES 



Compute grade 
cijuivalency scores & 
determine % meeting 
Ihe criteria. ^ 

Tally of raw scores 



Compute gra'de 
qujvalency scores & 
dcicrniinc^niceiing 
the criteria. , 



PROGRAM,, 
ACTIVITIES TO 
BE MONITORED 



# Individualized 
reading program 
aciivitieii 

I Usage of learning 
centers 

• Percent of time 
allotted to read' 
ing instruction 



• Individualized 
reading program 
activities 

f Usage of learning 
centers 

• Percent of time 
allotted to rcaJ- 
i.ig insiniction 



MONITORING 
DATES AND 
PERSONNEL 



Every month Sept. 
thru May ^ . 

Program Evaluator & 
Evaluation Assistant 



KEY ^ 

REPORTING 

DATES 



PERSONS/ 
AGENCIES 
TO RECEIVE 
REPORTS 



October 


Program Director 


Prerep<vi 


Advisory Committee 




Program Director 


Inleri III Report 






District Sup!., Pro- 


jiine 


gram Direcior.Board 


P'h;:i . 


of Educ. Advisory 


Iiif'V'iadon . 


Committee 



Every month Scpi, Ociobcr 

thru May | Prcicport 

Program Evaluaio' ^ ' Fcbniaiy 

livaluation \sm iiteriiriRepoil 



Prograrr i}l:?''(or 
'Advisor tnitlec 

Prjcram Director 



DiMict Supi.,Pro- 
,!nim Director, 



Ji,ic 

f^^'ilcM jn-rdofEd, 
'iiiiiimalio,, |,\jvjj;or^ 

I 'Jonimittee 

i 



USE TO BE 
MADE OF THE 
DATA/REPORTS 



BitSeline information 
to develop iniitruc* 
liona! plans. 

Progress of students 

■ i 

Determine \vheihcr f| 
to modify program. 



Modification 



Ba.scline information 
to develop instttic- 
tional plans. 

Progress of .students 
Program Modifica- 
tion 

Determine ^vhcther 
to continue program ' 



V 
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INFORMATION ABOUT EIP MATERIALS AND WORKSHOPS 

Informalion about ordering Evaluation Improvemen: Program materials, about Evaluation Improvemenl Program workshops that use 
these materials, or about making arrangements for specially schediiled EIP workshops for local, regional, or state inservice training 
programs may be obtained by writing or telephoning the Evaluation Improvement Program ai Educational Testing Service, Room 
P-069, Princeton, NJ 08540, (609) 92 1 -9000 or at any of its regional .offices listed below, 



REGIONAL OFFICES OF EDUCATIONAL TESTING SERVICE 



m Peachtree'Road, NE 
Suite 1040 

Atlanta, Georgia J0326 
(404) 262-7634 

3724 Jefferson, Suite 100 
Austin, Texas 78731 ■ 
1512) 452-8817 

!947 Center Street 
Berkeley, California 94704 
(415) 849.0950 



960 Grave Street 
Evanston, Illinois 60201 
(312) 869-7700 

GPOBox 1271 

San Juan, Puerto Rico 00936 

(809) 763-3636. 3640, or 3760 

One Dupont Circle 
Suite 310 

Washington, D,C, 20036 
(202) 296-5930 



2200 Merlon Avenue 
Room 216 

Los Angeles, California 90041 
(213) 254-5236 



2 Sun Life Executive Park 
100 Worcester Road 
Wellcsley Hills 
(617) 235-8861 or 8860 



Massachusetts 02181 
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